Centralized Coded Caching with User Cooperation

In this paper, we consider the coded-caching broadcast network with user cooperation, where a server connects with multiple users and the users can cooperate with each other through a cooperation network. We propose a centralized coded caching scheme based on a new deterministic placement strategy and a parallel delivery strategy. It is shown that the new scheme optimally allocate the communication loads on the server and users, obtaining cooperation gain and parallel gain that greatly reduces the transmission delay. Furthermore, we show that the number of users who parallelly send information should decrease when the users' caching size increases. In other words, letting more users parallelly send information could be harmful. Finally, we derive a constant multiplicative gap between the lower bound and upper bound on the transmission delay, which proves that our scheme is order optimal.Comment: 9 pages, submitted to ITW201

Cascaded Code Distributed Computing With Low Complexity and Improved Flexibility

Coded distributed computing (CDC), proposed by Li et al., offers significant potential for reducing the communication load in MapReduce computing systems. In the setting of the cascaded CDC that consisting of $K$ nodes, $N$ input files, and $Q$ output functions, the objective is to compute each output function through $s\geq 1$ nodes with a computation load $r\geq 1$, enabling the application of coding techniques during the Shuffle phase to achieve minimum communication load. However, a significant limitation in most existing cascaded CDC schemes is their demand for splitting the original data into an exponentially growing number of input files and requiring an exponentially large number of output functions, which imposes stringent requirements for implementation. In this paper, we focus on the cascaded case of $K/s\in\mathbb{N}$, deliberately designing the strategy of data placement and output functions assignment based on a grouping method, such that a low-complexity Shuffle strategy is achievable. The main advantages of the proposed scheme include: 1) the multicast gains equal to $(r+s-1)(1-1/s)$ and $r+s-1$ which is approximate to $r+s-1$ when $s$ is relatively large, and the communication load is quite approximate to or surprisingly better than the optimal state-of-the-art scheme proposed by Li et al.; 2) the proposed scheme requires significantly less number of input files and output functions; 3) all the operations are implemented over the minimum binary field $\mathbb{F}_2$ in the one-shot fashion. Finally, we derive a new converse bound for the cascaded CDC framework, under the given strategies of data placement and output functions assignment. We demonstrate that the communication load of the proposed scheme is order optimal within a factor of $2$; and is also approximately optimal when $K$ is sufficiently large for a given $r$

Multi-access Coded Caching with Optimal Rate and Linear Subpacketization under PDA and Consecutive Cyclic Placement

This work considers the multi-access caching system proposed by Hachem et al., where each user has access to L neighboring caches in a cyclic wrap-around fashion. We first propose a placement strategy called the consecutive cyclic placement, which achieves the maximal local caching gain. Then under the consecutive cyclic placement, we derive the optimal coded caching gain from the perspective of Placement Delivery Array (PDA), thus obtaining a lower bound on the rate of PDA. Finally, under the consecutive cyclic placement, we construct a class of PDA, leading to a multi-access coded caching scheme with linear subpacketization, which achieves our derived lower bound for some parameters; while for other parameters, the achieved coded caching gain is only 1 less than the optimal one. Analytical and numerical comparisons of the proposed scheme with existing schemes are provided to validate the performance.Comment: 30 pages, 7 figure

Hierarchical Cache-Aided Linear Function Retrieval with Security and Privacy Constraints

The hierarchical caching system where a server connects with multiple mirror sites, each connecting with a distinct set of users, and both the mirror sites and users are equipped with caching memories has been widely studied. However all the existing works focus on single file retrieval, i.e., each user requests one file, and ignore the security and privacy threats in communications. In this paper we investigate the linear function retrieval problem for hierarchical caching systems with content security and demand privacy, i.e., each user requests a linear combination of files, and meanwhile the files in the library are protected against wiretappers and users' demands are kept unknown to other users and unconnected mirror sites. First we propose a new combination structure named hierarchical placement delivery array (HPDA), which characterizes the data placement and delivery strategy of a coded caching scheme. Then we construct two classes of HPDAs. Consequently two classes of schemes with or without security and privacy are obtained respectively where the first dedicates to minimizing the transmission load for the first hop and can achieve the optimal transmission load for the first hop if ignoring the security and privacy constraints; the second has more flexible parameters on the memory sizes and a lower subpacketization compared with the first one, and achieves a tradeoff between subpacketization and transmission loads.Comment: arXiv admin note: substantial text overlap with arXiv:2205.0023

DPFormer: Learning Differentially Private Transformer on Long-Tailed Data

The Transformer has emerged as a versatile and effective architecture with broad applications. However, it still remains an open problem how to efficiently train a Transformer model of high utility with differential privacy guarantees. In this paper, we identify two key challenges in learning differentially private Transformers, i.e., heavy computation overhead due to per-sample gradient clipping and unintentional attention distraction within the attention mechanism. In response, we propose DPFormer, equipped with Phantom Clipping and Re-Attention Mechanism, to address these challenges. Our theoretical analysis shows that DPFormer can reduce computational costs during gradient clipping and effectively mitigate attention distraction (which could obstruct the training process and lead to a significant performance drop, especially in the presence of long-tailed data). Such analysis is further corroborated by empirical results on two real-world datasets, demonstrating the efficiency and effectiveness of the proposed DPFormer

One-Bit Byzantine-Tolerant Distributed Learning via Over-the-Air Computation

Distributed learning has become a promising computational parallelism paradigm that enables a wide scope of intelligent applications from the Internet of Things (IoT) to autonomous driving and the healthcare industry. This paper studies distributed learning in wireless data center networks, which contain a central edge server and multiple edge workers to collaboratively train a shared global model and benefit from parallel computing. However, the distributed nature causes the vulnerability of the learning process to faults and adversarial attacks from Byzantine edge workers, as well as the severe communication and computation overhead induced by the periodical information exchange process. To achieve fast and reliable model aggregation in the presence of Byzantine attacks, we develop a signed stochastic gradient descent (SignSGD)-based Hierarchical Vote framework via over-the-air computation (AirComp), where one voting process is performed locally at the wireless edge by taking advantage of Bernoulli coding while the other is operated over-the-air at the central edge server by utilizing the waveform superposition property of the multiple-access channels. We comprehensively analyze the proposed framework on the impacts including Byzantine attacks and the wireless environment (channel fading and receiver noise), followed by characterizing the convergence behavior under non-convex settings. Simulation results validate our theoretical achievements and demonstrate the robustness of our proposed framework in the presence of Byzantine attacks and receiver noise.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl